Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance for Rows: dropRight, partitionBy & sortBy #676

Merged
merged 2 commits into from
Oct 30, 2023

Conversation

stloyd
Copy link
Member

@stloyd stloyd commented Oct 30, 2023

Change Log

Added

Fixed

Changed

  • Improve performance for `Rows`: `dropRight`, `partitionBy` & `sortBy`

Removed

Deprecated

Security


Description

@github-actions
Copy link
Contributor

github-actions bot commented Oct 30, 2023

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+------------------+-------------------+
| benchmark             | subject           | revs | its | mem_peak         | mode              |
+-----------------------+-------------------+------+-----+------------------+-------------------+
| AvroExtractorBench    | bench_extract_10k | 5    | 3   | 44.121mb +0.00%  | 518.864ms +25.19% |
| CSVExtractorBench     | bench_extract_10k | 5    | 3   | 14.005mb +0.00%  | 386.843ms +13.69% |
| JsonExtractorBench    | bench_extract_10k | 5    | 3   | 18.667mb +0.00%  | 741.082ms +11.26% |
| ParquetExtractorBench | bench_extract_10k | 5    | 3   | 237.791mb +0.00% | 1.125s +23.56%    |
| TextExtractorBench    | bench_extract_10k | 5    | 3   | 7.275mb +0.00%   | 17.087ms +23.70%  |
| XmlExtractorBench     | bench_extract_10k | 5    | 3   | 7.622mb +0.00%   | 670.996ms +21.03% |
+-----------------------+-------------------+------+-----+------------------+-------------------+
Transformers
+-----------------------------+--------------------------+------+-----+-----------------+------------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode             |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 87.060mb +0.00% | 89.321ms +58.98% |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+
Loaders
+--------------------+----------------+------+-----+------------------+-------------------+
| benchmark          | subject        | revs | its | mem_peak         | mode              |
+--------------------+----------------+------+-----+------------------+-------------------+
| AvroLoaderBench    | bench_load_10k | 5    | 3   | 219.954mb +0.00% | 846.266ms +27.59% |
| CSVLoaderBench     | bench_load_10k | 5    | 3   | 47.138mb +0.00%  | 87.801ms +26.71%  |
| JsonLoaderBench    | bench_load_10k | 5    | 3   | 88.578mb +0.00%  | 84.633ms +32.05%  |
| ParquetLoaderBench | bench_load_10k | 5    | 3   | 281.199mb +0.00% | 1.077s +12.85%    |
| TextLoaderBench    | bench_load_10k | 5    | 3   | 16.564mb +0.00%  | 37.550ms +5.46%   |
+--------------------+----------------+------+-----+------------------+-------------------+
Building Blocks
+-------------------------+----------------------------+------+-----+-----------------+-------------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode              |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.692mb -0.01% | 4.044ms +34.74%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.484mb -0.01% | 194.734ms +16.91% |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 59.009mb -0.01% | 21.084ms +26.57%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.831mb -0.01% | 3.312ms +27.34%   |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 61.933mb -3.41% | 3.316ms +22.81%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 59.045mb -0.01% | 4.425ms +25.44%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.574mb -0.01% | 23.357ms +33.20%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.574mb -0.01% | 22.754ms +29.33%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.644mb -0.01% | 2.482μs -37.95%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.644mb -0.01% | 0.494μs -38.26%   |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.878mb -0.01% | 14.669ms +8.77%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.398mb -0.01% | 68.510ms +23.42%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 60.094mb -0.01% | 3.167ms +8.50%    |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.363mb -0.01% | 46.161ms +22.76%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 62.195mb -0.01% | 7.769ms +21.36%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.644mb -0.01% | 58.598ms +29.32%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.644mb -0.01% | 54.040ms +18.99%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.644mb -0.01% | 60.586ms +32.54%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.919mb -0.01% | 10.406ms +6.32%   |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.644mb -0.01% | 41.549ms +15.34%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.644mb -0.01% | 27.499μs +27.31%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.645mb -0.01% | 32.221μs +19.64%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.484mb -0.01% | 200.225ms +19.18% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | ERR             | 180.743ms +0.00%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | ERR             | 90.865ms +0.00%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | ERR             | 23.546ms +0.00%   |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+

@stloyd stloyd force-pushed the feature/rows-perf-part2 branch from 9072edf to a7126af Compare October 30, 2023 17:46
@stloyd stloyd changed the title Improve performance for Rows: dropRight, filter, map, partitionBy & sortBy Improve performance for Rows: dropRight, filter, partitionBy & sortBy Oct 30, 2023
@stloyd stloyd force-pushed the feature/rows-perf-part2 branch from a7126af to 726f150 Compare October 30, 2023 18:34
@github-actions github-actions bot added size: S and removed size: XS labels Oct 30, 2023
@stloyd stloyd force-pushed the feature/rows-perf-part2 branch from 548fe61 to 1ca68fd Compare October 30, 2023 19:29
@stloyd stloyd changed the title Improve performance for Rows: dropRight, filter, partitionBy & sortBy Improve performance for Rows: dropRight, partitionBy & sortBy Oct 30, 2023
@stloyd stloyd force-pushed the feature/rows-perf-part2 branch from 1ca68fd to 26d2f76 Compare October 30, 2023 19:44
@stloyd stloyd marked this pull request as ready for review October 30, 2023 19:48
@stloyd stloyd merged commit 72050d6 into flow-php:1.x Oct 30, 2023
17 checks passed
@stloyd stloyd deleted the feature/rows-perf-part2 branch October 30, 2023 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants